Skip to content

test(parity): compare against oold-js instead of oold-schema's scripts - #144

Merged
simontaurus merged 1 commit into
mainfrom
parity/reference-from-oold-js
Aug 30, 2026
Merged

test(parity): compare against oold-js instead of oold-schema's scripts#144
simontaurus merged 1 commit into
mainfrom
parity/reference-from-oold-js

Conversation

@simontaurus

@simontaurus simontaurus commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Closes #136. Pairs with OO-LD/oold-schema#156, which removes the scripts this stops pointing at.

validate.mjs, schema_to_frame.mjs and pattern_lint.mjs now live in OO-LD/oold-js, extracted unchanged and tagged v0.1.0.

Change

  • New reference_dir fixture reading OOLD_JS_DIR, alongside upstream_dir for OOLD_SCHEMA_DIR. The corpus and the reference are different repositories now and a machine can have either without the other.
  • A reference fixture runs the validator and both subprocess call sites collapse into it. It passes --meta <corpus>/meta, so both implementations read one file set and a disagreement can only come from the code - oold-js vendors no meta-schemas.
  • CI clones oold-js at a pinned tag and npm cis it. The corpus stays on main: pinning the reference makes a change to it deliberate, while leaving the corpus moving is what lets parity notice a specification change the fixture snapshot cannot.
  • CLAUDE.md, docs/architecture.md and docs/how-to/validation.md updated; the pin they cited (validate.mjs at v1.0.0-rc.2) no longer exists.

Without OOLD_JS_DIR the two verdict-comparing tests skip and the four corpus-only tests still run.

Verification

  • both variables set: 6 passed
  • OOLD_SCHEMA_DIR only: 4 passed, 2 skipped, naming OOLD_JS_DIR in the skip reason
  • pytest: 565 passed, 9 skipped
  • make docs-test: no issues

@simontaurus
test(parity): compare against oold-js instead of oold-schema's scripts
03e8526 
The reference implementation moved to OO-LD/oold-js, so the corpus and
the implementation checking it are now separate checkouts: OOLD_SCHEMA_DIR
for the examples and meta-schemas, OOLD_JS_DIR for the reference.

CI pins the reference by tag and leaves the corpus tracking main, so a
reference change is adopted deliberately while a specification change is
still noticed on the next run.

Closes #136
@github-actions

github-actions Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Release preview

No version bump from the current commits (stays at v0.18.2). Use conventional commit types (feat, fix, ...) to trigger a release.

Changelog preview (truncated) 




Preview via python-semantic-release and conventional commits.



      		
    

  





        

            

            

              
            

        

      


      

            
  
  

    
  
    
    

  

  

  


    












      

  
          

  

    
  


  

      


    @simontaurus
simontaurus



    mentioned this pull request
    
      Aug 30, 2026
    





  


  

      
   Merged

  







  









      

  
      



  

  @github-actions






  



    

      

  

    

      
          
      

      
            github-actions
    Bot

      

      

      commented


        Aug 30, 2026



      
    


  


  

    

      

          
    
    

  


        
            
  
      
              Copy link

  

            
  
      
              Copy Markdown

  

        
      

    


    

        

  
    Contributor


        

    

  





      


        



  
    

      
          
📊 Benchmark Results



Click to see benchmark comparison

📊 Benchmark Comparison (threshold: 1.3x)
============================================================

➖ Unchanged (within threshold):
  ➖ test_simple_dict_document_store: 0.0017s → 0.0017s (+1.5%)
  ➖ test_sqlite_document_store: 0.0019s → 0.0019s (-0.6%)
  ➖ test_local_sparql_store: 0.0385s → 0.0399s (+3.6%)
  ➖ test_oneof_subschema: 0.0594s → 0.0597s (+0.4%)
  ➖ test_enum_docstrings: 0.0508s → 0.0498s (-1.9%)
  ➖ test_subclass_inheritance: 0.0560s → 0.0550s (-1.7%)
  ➖ test_class_hierarchy: 0.0536s → 0.0508s (-5.2%)
  ➖ test_core[v1]: 0.0388s → 0.0400s (+3.1%)
  ➖ test_core[v2]: 0.0445s → 0.0443s (-0.6%)
  ➖ test_schema_generation[v1]: 0.0017s → 0.0017s (-0.4%)
  ➖ test_schema_generation[v2]: 0.0028s → 0.0028s (+0.4%)
  ➖ test_simple_json: 0.0007s → 0.0007s (+0.4%)
  ➖ test_complex_graph: 0.0016s → 0.0016s (-1.9%)

============================================================
Summary: 0 regressions, 0 improvements, 13 unchanged
============================================================

✅ No significant performance regressions





Threshold: 1.3x (30% slower triggers a regression warning)


Note: Benchmarks are informational only and won't fail the build.


💡 Tip: Download the benchmark-results artifact for detailed JSON data


      		
    

  





        

            

            

              
            

        

      


      

            
  
  

    
  
    
    

  

  

  


    












      

  
      



  

  @codecov






  



    

      

  

    

      
          
      

      
            codecov
    Bot

      

      

      commented


        Aug 30, 2026



      
    


  


  

    

      

          
    
    

  


        
            
  
      
              Copy link

  

            
  
      
              Copy Markdown

  

        
      

    


    

        



        

    

  





      


        



  
    

      
          
Codecov Report


✅ All modified and coverable lines are covered by tests.


📢 Thoughts on this report? Let us know!


      		
    

  





        

            

            

              
            

        

      


      

            
  
  

    
  
    
    

  

  

  


    












      

  
          
 
  

    
  

  


        
    

        

              @simontaurus


        

      

              simontaurus



            merged commit 42d0d74
          into

          
  
    main
  


        Aug 30, 2026

          

            21 checks passed
          

      

    


      

        
  
          
  
    
    


  
      

  







  

  
  

  
    

        

              @simontaurus


        

      

            simontaurus


        
    deleted the
    
      
        parity/reference-from-oold-js
    
    branch

        August 30, 2026 02:42        
        
      

    





  

  

    
  


  

      


  This was referenced Aug 31, 2026





  


  

      

  
  Closed

  






      







  


  

      
   Merged

  






      







  


  

      

  
  Open

  






  













  
  

    

      

  










  








  
      

  
    Sign up for free
    to join this conversation on GitHub.
    Already have an account?
    Sign in to comment


  










    
        

  

    
    

    

  

    Reviewers
  


    

    No reviews







  
      

  


      
  

    Assignees
  



        


    No one assigned







  
        


  


  

    Labels
  



    

    None yet







  
        

  

    

        

    Projects
  


        




    None yet




  



  
        


  

    
  

    Milestone
  


      No milestone





    
      
          


  

    

      
        

          
  

    Development
  



            


Successfully merging this pull request may close these issues.



  
          
      Repoint and pin the parity clone when validate.mjs moves to oold-js



  
  



      
    

  




      

  
  
        

  

    

      1 participant
    

    

        
          @simontaurus